<html>
<head><meta charset="utf-8"><title>ReadBuf RFC · wg-async-foundations · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/index.html">wg-async-foundations</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html">ReadBuf RFC</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="195615902"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195615902" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195615902">(Apr 28 2020 at 19:44)</a>:</h4>
<p>Speaking of uninitialized buffers, I finally started writing up that RFC. I have adjusted things a little bit since that dropbox paper/discussion with niko. The ReadBuf type now tracks both the initialization cursor and a read cursor, which avoids some weird edge cases around trusted and untrusted values from the impl: <a href="https://github.com/sfackler/rfcs/blob/read-buf/text/0000-read-buf.md" title="https://github.com/sfackler/rfcs/blob/read-buf/text/0000-read-buf.md">https://github.com/sfackler/rfcs/blob/read-buf/text/0000-read-buf.md</a></p>



<a name="195618124"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195618124" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195618124">(Apr 28 2020 at 19:59)</a>:</h4>
<p>definitely happy to take feedback/prs on the rfc if people feel like it, btw :)</p>



<a name="195646119"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646119" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646119">(Apr 29 2020 at 00:29)</a>:</h4>
<p><span class="user-mention" data-user-id="243558">@Steven Fackler</span> splitting this into its own topic for discussion</p>



<a name="195646307"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646307" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646307">(Apr 29 2020 at 00:32)</a>:</h4>
<p><span class="user-mention" data-user-id="243558">@Steven Fackler</span> do you think there's a way to avoid adding the second usize for <code>written</code> vs <code>initialized</code> bytes?</p>



<a name="195646327"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646327" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646327">(Apr 29 2020 at 00:33)</a>:</h4>
<p>So originally I didn't have the written tracked in the buf at all: <a href="https://paper.dropbox.com/doc/IO-Buffer-Initialization--Ay0GuctVvpVkBU1tnhs3EGe5Ag-MvytTgjIOTNpJAS6Mvw38" title="https://paper.dropbox.com/doc/IO-Buffer-Initialization--Ay0GuctVvpVkBU1tnhs3EGe5Ag-MvytTgjIOTNpJAS6Mvw38">https://paper.dropbox.com/doc/IO-Buffer-Initialization--Ay0GuctVvpVkBU1tnhs3EGe5Ag-MvytTgjIOTNpJAS6Mvw38</a></p>



<a name="195646346"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646346" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646346">(Apr 29 2020 at 00:33)</a>:</h4>
<p>yeah, that's part of what prompted my question</p>



<a name="195646348"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646348" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646348">(Apr 29 2020 at 00:33)</a>:</h4>
<p>where read_buf returns the number of bytes read just like the normal read function does</p>



<a name="195646349"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646349" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646349">(Apr 29 2020 at 00:33)</a>:</h4>
<p>however</p>



<a name="195646356"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646356" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646356">(Apr 29 2020 at 00:33)</a>:</h4>
<p>this introduces some subtle issues when dealing with unsafe code</p>



<a name="195646418"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646418" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646418">(Apr 29 2020 at 00:34)</a>:</h4>
<p>you can't trust that the returned number of bytes read is correct</p>



<a name="195646423"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646423" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646423">(Apr 29 2020 at 00:34)</a>:</h4>
<p>in particular it could be larger than the number of initialized bytes</p>



<a name="195646456"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646456" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646456">(Apr 29 2020 at 00:35)</a>:</h4>
<p>so when working with unsafe code around read bufs, there are some subtle asserts you need to add</p>



<a name="195646459"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646459" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646459">(Apr 29 2020 at 00:35)</a>:</h4>
<p>e.g. <a href="https://github.com/sfackler/read-buf/blob/master/src/read.rs#L36-L39" title="https://github.com/sfackler/read-buf/blob/master/src/read.rs#L36-L39">https://github.com/sfackler/read-buf/blob/master/src/read.rs#L36-L39</a> and <a href="https://github.com/sfackler/read-buf/blob/master/src/read.rs#L82-L85" title="https://github.com/sfackler/read-buf/blob/master/src/read.rs#L82-L85">https://github.com/sfackler/read-buf/blob/master/src/read.rs#L82-L85</a></p>



<a name="195646508"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646508" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646508">(Apr 29 2020 at 00:36)</a>:</h4>
<p>but if we move the "bytes read" tracking inside of the ReadBuf, we can lean on ReadBuf to make sure it's asserting properly for us</p>



<a name="195646514"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646514" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646514">(Apr 29 2020 at 00:36)</a>:</h4>
<p>I see, so we need the implementation of the ReadBuf to enforce correctness when being used from potentially incorrect code</p>



<a name="195646519"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646519" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646519">(Apr 29 2020 at 00:36)</a>:</h4>
<p>it also makes read_exact cleaner since you can just repeatedly pass the same buffer in without needing to reconstruct it each time</p>



<a name="195646533"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646533" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646533">(Apr 29 2020 at 00:37)</a>:</h4>
<p>I mean, you don't <em>need to</em> have ReadBuf deal with it</p>



<a name="195646548"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646548" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646548">(Apr 29 2020 at 00:37)</a>:</h4>
<p>but it just seemed like a footgun to have a mix of trusted and untrusted values</p>



<a name="195646559"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646559" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646559">(Apr 29 2020 at 00:37)</a>:</h4>
<p>it does make "did I read zero bytes" checks a bit awkward though</p>



<a name="195646825"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646825" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646825">(Apr 29 2020 at 00:42)</a>:</h4>
<p>ok, makes sense. that might be the right tradeoff to make</p>



<a name="195646963"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646963" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646963">(Apr 29 2020 at 00:45)</a>:</h4>
<p>one idea that comes to mind is returning the bytes read, but wrapping it a newtype that sort of advertises / forces you to acknowledge the fact that it's untrusted, but I can't think of one that's ergonomic for the safe case and loud for the unsafe.</p>



<a name="195646994"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646994" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646994">(Apr 29 2020 at 00:45)</a>:</h4>
<p>well, maybe I <em>can</em> think of one</p>



<a name="195646996"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195646996" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195646996">(Apr 29 2020 at 00:45)</a>:</h4>
<p>but I'm not sure if it'd be worth the complexity in the API</p>



<a name="195647632"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195647632" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195647632">(Apr 29 2020 at 00:53)</a>:</h4>
<p>we could have read_buf still just return io::Result&lt;usize&gt;, but then you have a trusted number of bytes read and an untrusted number which also seems like a footgun</p>



<a name="195647731"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195647731" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195647731">(Apr 29 2020 at 00:54)</a>:</h4>
<p>..the return value here is the untrusted number, correct?</p>



<a name="195647924"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195647924" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195647924">(Apr 29 2020 at 00:57)</a>:</h4>
<p>I guess what I'm saying is, you could change the return type to <code>io::Result&lt;ReadBufCursor&gt;</code> or something</p>



<a name="195647975"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195647975" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195647975">(Apr 29 2020 at 00:58)</a>:</h4>
<p>and then force the caller to bring the cursor and the readbuf together to do anything useful</p>



<a name="195648028"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195648028" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195648028">(Apr 29 2020 at 00:58)</a>:</h4>
<p>I don't actually know if that works though, I haven't written out an example yet</p>



<a name="195648159"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195648159" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195648159">(Apr 29 2020 at 01:00)</a>:</h4>
<p>and in the vectored case it's more unclear.. maybe you could update the cursor as you consume the bytes from each underlying buf, or something</p>



<a name="195648300"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195648300" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195648300">(Apr 29 2020 at 01:02)</a>:</h4>
<p>(but then that would impose a constraint that you read the vectored bufs in order)</p>



<a name="195648377"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195648377" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195648377">(Apr 29 2020 at 01:02)</a>:</h4>
<p>I think it'd be far more useful if I sat down to write some code than speak in generalities</p>



<a name="195650177"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195650177" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195650177">(Apr 29 2020 at 01:30)</a>:</h4>
<p>yeah the return value is the untrusted one</p>



<a name="195650739"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195650739" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195650739">(Apr 29 2020 at 01:42)</a>:</h4>
<p>I think you'd have a hard time making ReadBufCursor safe since you'd need to prevent a malicious Read impl from creating a large one with a different reader and returning that</p>



<a name="195996033"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/ReadBuf%20RFC/near/195996033" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/ReadBuf.20RFC.html#195996033">(May 01 2020 at 18:55)</a>:</h4>
<p>Ah, my thinking was that ReadBufCursor would remain untrusted (it's a newtype around the return value we have today), but that we'd be encoding that in the API instead of in documentation. So you'd be forced to go through an asserting path to "unwrap" the value, essentially.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>